Software development usually requires testers to evaluate the quality of a software product in many ways, including any content such as images incorporated in a software build or generated by execution of a software build. Traditional testing of such images may involve verifying that every pixel of a test candidate image is identical to the corresponding pixel of a master image. A typical validation algorithm may directly compare each pixel of a test candidate image to a corresponding pixel in the master image and then decide to pass or fail the test candidate image based upon the number of direct pixel comparison failures. The problem with this approach is that the comparison may fail from numerous slight differences that may not be visible by human perception.
Another problem with this approach is that it does not account for noise that may be introduced into an image such as by disk drive controllers in accessing the image from persistent storage or by rendering drivers in rendering an image retrieved from storage. Based on the specifics of the system, some other common sources of noise may include rounding errors due to the high precision of the computations, differences in bit representation of the image format, visually detectable artifacts such as salt and pepper noise from spurious pixels, information missing from the image or extraneous information added to the image, and so forth. In any of these cases, a direct pixel comparison approach may often fail where noise has been introduced in the image.
Additionally, differences in the implementations of graphics hardware and software drivers, including renderers, may create differences in two rendered images. For example, rounded corners in an image may be rendered as square corners. As another example, firmware in an implementation may render red as a shade of red or reddish. Again, a direct pixel comparison approach may often fail from the accumulation of defects arising from differences that may be generated by various implementations of graphics hardware and software.
What is needed is a more flexible and adjustable system for comparing images. Such a system should be able to compensate for noise introduced into an image as well as allow for adjustable control of the resolution of an image.